Anomaly Detection in Lidar Data by Combining Supervised and Self-Supervised Methods

To enable safe autonomous driving, a reliable and redundant perception of the environment is required. In the context of autonomous vehicles, the perception is mainly based on machine learning models that analyze data from various sensors such as camera, Radio Detection and Ranging (radar), and Light Detection and Ranging (lidar). Since the performance of the models depends significantly on the training data used, it is necessary to ensure perception even in situations that are difficult to analyze and deviate from the training dataset. These situations are called corner cases or anomalies. Motivated by the need to detect such situations, this thesis presents a new approach for detecting anomalies in lidar data by combining Supervised (SV) and Self-Supervised (SSV) models. In particular, inconsistent point-wise predictions between a SV and a SSV part serve as an indication of anomalies arising from the models used themselves, e.g., due to lack of knowledge. The SV part is composed of a SV semantic segmentation model and a SV moving object segmentation model, which together assign a semantic motion class to each point of the point cloud. Based on the definition of semantic motion classes, a first motion label, denoting whether the point is static or dynamic, is predicted for each point. The SSV part mainly consists of a SSV scene flow model and a SSV odometry model and predicts a second motion label for each point. Thereby, the scene flow model estimates a displacement vector for each point, which, using the odometry information of the odometry model, represents only a point’s own induced motion. A separate quantitative analysis of the two parts and a qualitative analysis of the anomaly detection capabilities by combining the two parts are performed. In the qualitative analysis, the frames are classified into four main categories, namely correctly consistent, incorrectly consistent, anomalies detected by the SSV part, and anomalies detected by the SV part. In addition, weaknesses were identified in both the SV part and the SSV part